In imaging encoding technology, transformed coefficients are quantized by using a quantization matrix. The quantized result is then subjected to entropy encoding to produce a compressed bitstream. The quality of the encoded image can be controlled by adjusting the coefficients in the quantization matrix.
The image may include a stationary picture, a frame of moving pictures, a residual picture of two adjacent pictures of moving pictures, or a target picture resulted from performing computation on any one of moving pictures.
During encoding procedure, the quantization of transformed coefficients is normally carried out via a quantization matrix. For example, the quantization can be done using the following equation:
                              Q          ⁡                      (                          i              ,              j                        )                          =                  [                                    Coe              ⁡                              (                                  i                  ,                  j                                )                                                    QM              ⁡                              (                                  i                  ,                  j                                )                                              ]                                    (        1        )            where, Coe(i, j) is the value of the (i, j) pixel of a transformed picture block, which is referred to as transformed coefficient in short; QM is a quantization matrix, QM (i, j) is a coefficient value of the quantization matrix, Q(i, j) stands for the quantized and rounded value of the transformed coefficient Coe(i, j), in which Q(i, j) is referred to as quantized coefficient in short, and [●] is a round operation.
Details of different regions in a picture represent different frequency features of the picture, and the human's eyes have different subjective perception for different parts of a picture; thus, different quantization methods conforming to the human's eyes' characteristic should be applied respectively according to different picture contents.
Presently, quantization of encoded pictures is implemented with a constant quantization matrix in the image encoding standards of JPEG (Joint Photographic Experts Group), MPEG (Moving Picture Experts Group), MPEG-2 and MPEG-4, and so on. In these standards, the quantization matrix is set in the picture header for JPEG, while for MPEG-1, MPEG-2 and MPEG-4, the quantization matrix is set in the sequence header. Therefore, for a sequence of pictures, each sequence has a quantization matrix according to the MPEG standards. In other words, quantization of the pictures in the entire sequence is performed with a single fixed quantization matrix.
When a human watches a picture, the evaluation of the picture quality is achieved and based on the subjective picture quality capable of being perceived by the human's eyes. Thus, a better subjective picture quality can be obtained by employing quantization methods conforming to the human visual characteristics. This means that an appropriate quantization matrix has to be selected for the different pictures in a sequence to obtain a satisfying subjective quality for the encoded picture sequence.
However, picture contents in a sequence are not always the same, rather, variations and details of each picture in a picture sequence are significantly different from each other. As a result, applying a single quantization matrix to all the pictures in the sequence cannot achieve the best subjective picture quality for the encoded pictures.
Therefore, in the H.264/AVC standard, both the sequence header and the picture header include user-defined quantization matrices, so as to select an appropriate quantization matrix at a picture level for different video pictures since picture contents in a sequence vary significantly.
Several representative quantization matrices in the currently image/video coding standards are described briefly below.
Quantization Matrix in JPEG
In the JPEG standard, there is only one size of discrete cosine transform (DCT), that is, an 8×8 transform, the quantization matrix is thus also an 8×8 matrix, and there are totally 64 quantization coefficients. Different quantization matrices are used for the luminance component and chroma component of a picture. The quantization matrix for the luminance component is:
      QM_Y    =          [              16        ,        11        ,        10        ,        16        ,        24        ,        40        ,        51        ,        61        ,        12        ,        12        ,        14        ,        19        ,        26        ,        58        ,        60        ,        55        ,        14        ,        13        ,        16        ,        24        ,        40        ,        57        ,        69        ,        56        ,        14        ,        17        ,        22        ,        29        ,        51        ,        87        ,        80        ,        62        ,        18        ,        22        ,        37        ,        56        ,        68        ,        109        ,        103        ,        77        ,        24        ,        35        ,        55        ,        64        ,        81        ,        104        ,        113        ,        92        ,        49        ,        64        ,        78        ,        87        ,        103        ,        121        ,        120        ,        101        ,        72        ,        92        ,        95        ,        98        ,        112        ,        100        ,        103        ,        99            ]        ;
The quantization matrix for the chroma component is:
  QM_C  =            [              17        ,        18        ,        24        ,        47        ,        99        ,        99        ,        99        ,        99        ,        18        ,        21        ,        26        ,        66        ,        99        ,        99        ,        99        ,        99        ,        24        ,        26        ,        56        ,        99        ,        99        ,        99        ,        99        ,        99        ,        47        ,        66        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99        ,        99            ]        .  
All the coefficients of the JPEG quantization matrices are set in the picture header, and there is one luminance quantization matrix and one chroma quantization matrix for each picture.
(2) MPEG-2
In the MPEG-2 standard, there is only one size of DCT, that is, an 8×8 transform, the quantization matrix is thus also an 8×8 matrix, and there are totally 64 quantization coefficients. Intra-frame quantization matrix and inter-frame quantization matrix are applied while encoding picture, where the intra-frame quantization matrix is:
      QM_INTRA    =          [              8        ,        16        ,        19        ,        22        ,        26        ,        27        ,        29        ,        34        ,        16        ,        16        ,        22        ,        24        ,        27        ,        29        ,        34        ,        37        ,        19        ,        22        ,        26        ,        27        ,        29        ,        34        ,        34        ,        38        ,        22        ,        22        ,        26        ,        27        ,        29        ,        34        ,        37        ,        40        ,        22        ,        26        ,        27        ,        29        ,        32        ,        35        ,        40        ,        48        ,        26        ,        27        ,        29        ,        32        ,        35        ,        40        ,        48        ,        58        ,        26        ,        27        ,        29        ,        34        ,        38        ,        46        ,        56        ,        69        ,        27        ,        29        ,        35        ,        38        ,        46        ,        56        ,        69        ,        83            ]        ;The inter-frame quantization matrix is:
  QC_INTER  =            [              16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16        ,        16            ]        .  
In MPEG-2, only one intra-frame quantization matrix and only one inter-frame quantization matrix are available for all the pictures in each sequence, and all of the 64 coefficients are set in the sequence header. MPEG-2 also allows for a user-defined quantization matrix, which is set in the sequence header extension.
(3) H.264/AVC
In the H.264/AVC standard, there are two DCT sizes: 8×8 and 4×4, and thus there are two sets of quantization matrices of 8×8 and 4×4 correspondingly. For the 8×8 quantization matrix, totally 64 coefficients are used for quantization scaling of different frequency components; for the 4×4 quantization matrix, totally 16 coefficients are used for quantization scaling of different frequency components.
In High Profile of H.264/AVC standard, quantization matrices exist in both the sequence header and picture header. There exists quantization matrices corresponding to the 4×4 and 8×8 blocks. Thus, H.264/AVC High Profile not only allows all the pictures in each sequence to have the same quantization matrix but also allows different pictures in the same sequence to have different quantization matrices, though only one quantization matrix can be used during encoding one image of the same sequence.
The quality of the encoded picture is thus flexibly controlled by adjusting the quantization matrix during the video encoding.
The quality control of an encoded image is carried out by adjusting the coefficients of quantization matrices in the sequence header and image header.
The quantization matrix has a size of M×N (M, N=2, 4, 8, 16 or other size). There are different quantization matrices for different transform sizes.
Presently, all the coefficient values or transformed values of the coefficient values in a quantization matrix are included in the sequence header or picture header during the video encoding. The coefficient values of the quantization matrix are the values corresponding to the transforming coefficient of the picture block. For each transformed coefficient of the picture block, there exists a corresponding quantization coefficient in the quantization matrix. Similarly, the transformed values of the quantization matrix coefficients refer to weighting/scaling values in quantization matrix or transformed values of weighting/scaling values in quantization matrix, such as differential values from a default quantization matrix.
It can be seen from the above quantization processes of the individual encoding method that, during the practical application, it is necessary to adjust a value corresponding to each frequency component in the quantization matrix so as to control the image quality, which increases the processing complexity. On the other hand, an overhead is caused when transferring the corresponding matrix coefficients, which affects the image encoding efficiency and transferring performance.